Articulated Surgical Tool Detection Using Virtually-Rendered Templates

Purpose We propose a system capable of detecting articulated surgical instruments without the use of assistive markers or manual initialization. Methods The algorithm can provide 3D pose using a combination of online and offline learning techniques along with prior geometric knowledge of the tool. It uses live kinematic data from the robotic system to render nearby poses on-thefly as virtual images and creates gradient orientation templates for fast matching into the real image. Prior appearance models of different material classes and projective invariance are used to reject false positives. Results Results are verified using in-vivo data recorded from the da Vinci R © robotic surgical system. The method detects successfully at a high correctness rate and a pyramid search method is proposed which reduces a brute-force method from 23 secs/frame down to 3 secs/frame. Conclusion We have shown a top-down approach to detect surgical tools within in-vivo video sequences and is capable of determining the pose and articulation by learning on-thefly from virtual renderings driven by real kinematic data. A. Reiter, P. K. Allen Dept. of Computer Science Columbia University 500 W. 120th Street, M.C. 0401 New York, NY 10027 Tel.: +1 212-939-7093 Fax: +1 212-666-0140 E-mail: areiter, [email protected] T. Zhao Intuitive Surgical, Inc. 1266 Kifer Road, Bldg. 101 Sunnyvale, CA 94086 Tel.: +1 408-523-2100 Fax: +1 408-523-1390 E-mail: [email protected]